Papers with out-of-domain evaluations
Using Linguistic Features to Improve the Generalization Capability of Neural Coreference Resolvers (D18-1)
Copied to clipboard
| Challenge: | Recent coreference resolvers have notable improvements on the CoNLL evaluation sets, but struggle to generalize properly to new datasets. |
| Approach: | They investigate the role of linguistic features in building more generalizable coreference resolvers . they show that employing features and subsets of their values that are informative for coreference resolution improves generalization . |
| Outcome: | The proposed system achieves state-of-the-art results on WikiCoref, compared with a system trained on CoNLL. |
Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) often produce inconsistent answers when faced with different phrasings of the same prompt. |
| Approach: | They propose an unsupervised method that flips the phrasings of prompts into a hard pseudo-label . they use Consensus Cross-Entropy to create a consensus, and representation alignment loss to pull lower-confidence predictors toward consensus . |
| Outcome: | The proposed method raises observed agreement by 11.62% and improves mean F1 by 8.94% on 11 datasets spanning four NLP tasks . |
From Reading to Compressing: Exploring the Multi-document Reader for Prompt Compression (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have recently exhibited performance gains owing to a wide variety of prompting techniques, including Retrieval-Augmented Generation (RAG), Chain-of-Thought (CoT), and In-Context Learning (ICL). |
| Approach: | They propose a prompt compression method that captures the global context without compromising semantic consistency while detouring the necessity of pseudo-labels for training the compressor. |
| Outcome: | Empirical results show that the proposed method retains key contexts while reducing the prompt length by 80%. |